Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use "float64" dtypes in DaskExecutor.fit_phase #378

Merged
merged 6 commits into from
Jul 26, 2024
Merged

Use "float64" dtypes in DaskExecutor.fit_phase #378

merged 6 commits into from
Jul 26, 2024

Conversation

rjzamora
Copy link
Contributor

@rjzamora rjzamora commented Jul 26, 2024 •
edited
Loading

Addresses NVTabular test failures with rapids-24.04 (pandas-2).

This PR does not "fix" the existing dtype-preservation problem:

            # TODO: constructing meta like this loses dtype information on the ddf
            # and sets it all to 'float64'. We should propagate dtype information along
            # with column names in the columngroup graph. This currently only
            # happens during intermediate 'fit' transforms, so as long as statoperators
            # don't require dtype information on the DDF this doesn't matter all that much

However, it makes pandas-2 behavior consistent with pandas<2.

@github-actions GitHub Actions
Copy link

Documentation preview

https://nvidia-merlin.github.io/core/review/pr-378

@rjzamora rjzamora added the bug Something isn't working label Jul 26, 2024
@rjzamora rjzamora changed the title Preserve dtypes in DaskExecutor.fit_phase Use "float64" dtypes in DaskExecutor.fit_phase Jul 26, 2024
benfred
benfred approved these changes Jul 26, 2024
View reviewed changes
Copy link
Member

@benfred benfred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@rjzamora rjzamora merged commit 6d396aa into NVIDIA-Merlin:main Jul 26, 2024
17 checks passed
@rjzamora rjzamora deleted the preserve-dtypes branch July 26, 2024 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@benfred benfred benfred approved these changes

Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@rjzamora @benfred